{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# How to use a chat model to call tools\n",
        "\n",
        ":::info Prerequisites\n",
        "\n",
        "This guide assumes familiarity with the following concepts:\n",
        "\n",
        "- [Chat models](/docs/concepts/#chat-models)\n",
        "- [LangChain Tools](/docs/concepts/#tools)\n",
        "\n",
        ":::\n",
        "\n",
        "```{=mdx}\n",
        ":::info\n",
        "We use the term tool calling interchangeably with function calling. Although\n",
        "function calling is sometimes meant to refer to invocations of a single function,\n",
        "we treat all models as though they can return multiple tool or function calls in \n",
        "each message.\n",
        ":::\n",
        "```\n",
        "\n",
        "Tool calling allows a chat model to respond to a given prompt by \"calling a tool\".\n",
        "While the name implies that the model is performing \n",
        "some action, this is actually not the case! The model generates the \n",
        "arguments to a tool, and actually running the tool (or not) is up to the user.\n",
        "For example, if you want to [extract output matching some schema](/docs/how_to/structured_output/) \n",
        "from unstructured text, you could give the model an \"extraction\" tool that takes \n",
        "parameters matching the desired schema, then treat the generated output as your final \n",
        "result.\n",
        "\n",
        "However, tool calling goes beyond [structured output](/docs/how_to/structured_output/)\n",
        "since you can pass responses to caled tools back to the model to create longer interactions.\n",
        "For instance, given a search engine tool, an LLM might handle a \n",
        "query by first issuing a call to the search engine with arguments. The system calling the LLM can \n",
        "receive the tool call, execute it, and return the output to the LLM to inform its \n",
        "response. LangChain includes a suite of [built-in tools](/docs/integrations/tools/) \n",
        "and supports several methods for defining your own [custom tools](/docs/how_to/custom_tools). \n",
        "\n",
        "Tool calling is not universal, but many popular LLM providers, including [Anthropic](https://www.anthropic.com/), \n",
        "[Cohere](https://cohere.com/), [Google](https://cloud.google.com/vertex-ai), \n",
        "[Mistral](https://mistral.ai/), [OpenAI](https://openai.com/), and others, \n",
        "support variants of a tool calling feature.\n",
        "\n",
        "LangChain implements standard interfaces for defining tools, passing them to LLMs, \n",
        "and representing tool calls. This guide will show you how to use them.\n",
        "\n",
        "## Passing tools to LLMs\n",
        "\n",
        "Chat models that support tool calling features implement a [`.bindTools()`](https://v02.api.js.langchain.com/classes/langchain_core_language_models_chat_models.BaseChatModel.html#bindTools) method, which \n",
        "receives a list of LangChain [tool objects](https://v02.api.js.langchain.com/classes/langchain_core_tools.StructuredTool.html) \n",
        "and binds them to the chat model in its expected format. Subsequent invocations of the \n",
        "chat model will include tool schemas in its calls to the LLM.\n",
        "\n",
        "Let's walk through a few examples:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "```{=mdx}\n",
        "import ChatModelTabs from \"@theme/ChatModelTabs\";\n",
        "\n",
        "<ChatModelTabs customVarName=\"llm\" providers={[\"anthropic\", \"openai\", \"mistral\", \"fireworks\"]} additionalDependencies=\"@langchain/core\" />\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "A number of models implement helper methods that will take care of formatting and binding different function-like objects to the model.\n",
        "Let's take a look at how we might take the following Zod function schema and get different models to invoke it:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {},
      "outputs": [],
      "source": [
        "import { z } from \"zod\";\n",
        "\n",
        "/**\n",
        " * Note that the descriptions here are crucial, as they will be passed along\n",
        " * to the model along with the class name.\n",
        " */\n",
        "const calculatorSchema = z.object({\n",
        "  operation: z\n",
        "    .enum([\"add\", \"subtract\", \"multiply\", \"divide\"])\n",
        "    .describe(\"The type of operation to execute.\"),\n",
        "  number1: z.number().describe(\"The first number to operate on.\"),\n",
        "  number2: z.number().describe(\"The second number to operate on.\"),\n",
        "});"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We can use the `.bindTools()` method to handle the conversion from LangChain tool to our model provider's specific format and bind it to the model (i.e., passing it in each time the model is invoked). Let's create a `DynamicStructuredTool` implementing a tool based on the above schema, then bind it to the model:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {},
      "outputs": [],
      "source": [
        "import { ChatOpenAI } from \"@langchain/openai\";\n",
        "import { DynamicStructuredTool } from \"@langchain/core/tools\";\n",
        "\n",
        "const calculatorTool = new DynamicStructuredTool({\n",
        "  name: \"calculator\",\n",
        "  description: \"Can perform mathematical operations.\",\n",
        "  schema: calculatorSchema,\n",
        "  func: async ({ operation, number1, number2 }) => {\n",
        "    // Functions must return strings\n",
        "    if (operation === \"add\") {\n",
        "      return `${number1 + number2}`;\n",
        "    } else if (operation === \"subtract\") {\n",
        "      return `${number1 - number2}`;\n",
        "    } else if (operation === \"multiply\") {\n",
        "      return `${number1 * number2}`;\n",
        "    } else if (operation === \"divide\") {\n",
        "      return `${number1 / number2}`;\n",
        "    } else {\n",
        "      throw new Error(\"Invalid operation.\");\n",
        "    }\n",
        "  }\n",
        "});\n",
        "\n",
        "const llmWithTools = llm.bindTools([calculatorTool]);"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now, let's invoke it! We expect the model to use the calculator to answer the question:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[\n",
            "  {\n",
            "    name: \"calculator\",\n",
            "    args: { operation: \"multiply\", number1: 3, number2: 12 },\n",
            "    id: \"call_Ri9s27J17B224FEHrFGkLdxH\"\n",
            "  }\n",
            "]\n"
          ]
        }
      ],
      "source": [
        "const res = await llmWithTools.invoke(\"What is 3 * 12\");\n",
        "\n",
        "console.log(res.tool_calls);"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "```{=mdx}\n",
        ":::tip\n",
        "See a LangSmith trace for the above [here](https://smith.langchain.com/public/14e4b50c-c6cf-4c53-b3ef-da550edb6d66/r).\n",
        ":::\n",
        "```\n",
        "\n",
        "We can see that the response message contains a `tool_calls` field when the model decides to call the tool. This will be in LangChain's standardized format."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The `.tool_calls` attribute should contain valid tool calls. Note that on occasion, \n",
        "model providers may output malformed tool calls (e.g., arguments that are not \n",
        "valid JSON). When parsing fails in these cases, the message will contain instances of \n",
        "of [InvalidToolCall](https://v02.api.js.langchain.com/types/langchain_core_messages_tool.InvalidToolCall.html) objects in the `.invalid_tool_calls` attribute. An `InvalidToolCall` can have \n",
        "a name, string arguments, identifier, and error message.\n",
        "\n",
        "### Streaming\n",
        "\n",
        "When tools are called in a streaming context, \n",
        "[message chunks](https://v02.api.js.langchain.com/classes/langchain_core_messages.BaseMessageChunk.html) \n",
        "will be populated with [tool call chunk](https://v02.api.js.langchain.com/types/langchain_core_messages_tool.ToolCallChunk.html)\n",
        "objects in a list via the `.tool_call_chunks` attribute. A `ToolCallChunk` includes \n",
        "optional string fields for the tool `name`, `args`, and `id`, and includes an optional \n",
        "integer field `index` that can be used to join chunks together. Fields are optional \n",
        "because portions of a tool call may be streamed across different chunks (e.g., a chunk \n",
        "that includes a substring of the arguments may have null values for the tool name and id).\n",
        "\n",
        "Because message chunks inherit from their parent message class, an \n",
        "[AIMessageChunk](https://v02.api.js.langchain.com/classes/langchain_core_messages.AIMessageChunk.html) \n",
        "with tool call chunks will also include `.tool_calls` and `.invalid_tool_calls` fields. \n",
        "These fields are parsed best-effort from the message's tool call chunks.\n",
        "\n",
        "Note that not all providers currently support streaming for tool calls. If this is the case for your specific provider, the model will yield a single chunk with the entire call when you call `.stream()`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[\n",
            "  {\n",
            "    name: \"calculator\",\n",
            "    args: \"\",\n",
            "    id: \"call_rGqPR1ivppYUeBb0iSAF8HGP\",\n",
            "    index: 0\n",
            "  }\n",
            "]\n",
            "[ { name: undefined, args: '{\"', id: undefined, index: 0 } ]\n",
            "[ { name: undefined, args: \"operation\", id: undefined, index: 0 } ]\n",
            "[ { name: undefined, args: '\":\"', id: undefined, index: 0 } ]\n",
            "[ { name: undefined, args: \"divide\", id: undefined, index: 0 } ]\n",
            "[ { name: undefined, args: '\",\"', id: undefined, index: 0 } ]\n",
            "[ { name: undefined, args: \"number\", id: undefined, index: 0 } ]\n",
            "[ { name: undefined, args: \"1\", id: undefined, index: 0 } ]\n",
            "[ { name: undefined, args: '\":', id: undefined, index: 0 } ]\n",
            "[ { name: undefined, args: \"308\", id: undefined, index: 0 } ]\n",
            "[ { name: undefined, args: ',\"', id: undefined, index: 0 } ]\n",
            "[ { name: undefined, args: \"number\", id: undefined, index: 0 } ]\n",
            "[ { name: undefined, args: \"2\", id: undefined, index: 0 } ]\n",
            "[ { name: undefined, args: '\":', id: undefined, index: 0 } ]\n",
            "[ { name: undefined, args: \"29\", id: undefined, index: 0 } ]\n",
            "[ { name: undefined, args: \"}\", id: undefined, index: 0 } ]\n",
            "[]\n"
          ]
        }
      ],
      "source": [
        "const stream = await llmWithTools.stream(\"What is 308 / 29\");\n",
        "\n",
        "for await (const chunk of stream) {\n",
        "  console.log(chunk.tool_call_chunks);\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Note that using the `concat` method on message chunks will merge their corresponding tool call chunks. This is the principle by which LangChain's various [tool output parsers](/docs/how_to/output_parser_structured/) support streaming.\n",
        "\n",
        "For example, below we accumulate tool call chunks:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[\n",
            "  {\n",
            "    name: \"calculator\",\n",
            "    args: { operation: \"subtract\", number1: 32993, number2: 2339 },\n",
            "    id: \"call_WMhL5X0fMBBZPNeyUZY53Xuw\"\n",
            "  }\n",
            "]\n"
          ]
        }
      ],
      "source": [
        "const streamWithAccumulation = await llmWithTools.stream(\"What is 32993 - 2339\");\n",
        "\n",
        "let final;\n",
        "for await (const chunk of streamWithAccumulation) {\n",
        "  if (!final) {\n",
        "    final = chunk;\n",
        "  } else {\n",
        "    final = final.concat(chunk);\n",
        "  }\n",
        "}\n",
        "\n",
        "console.log(final.tool_calls);"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Few shotting with tools\n",
        "\n",
        "You can give the model examples of how you would like tools to be called in order to guide generation by inputting manufactured tool call turns. For example, given the above calculator tool, we could define a new operator, `🦜`. Let's see what happens when we use it naively:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "It seems like you've used an emoji (🦜) in your expression, which I'm not familiar with in a mathematical context. Could you clarify what operation you meant by using the parrot emoji? For example, did you mean addition, subtraction, multiplication, or division?\n",
            "[]\n"
          ]
        }
      ],
      "source": [
        "const res = await llmWithTools.invoke(\"What is 3 🦜 12\");\n",
        "\n",
        "console.log(res.content);\n",
        "console.log(res.tool_calls);"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "It doesn't quite know how to interpret `🦜` as an operation. Now, let's try giving it an example in the form of a manufactured messages to steer it towards `divide`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[\n",
            "  {\n",
            "    name: \"calculator\",\n",
            "    args: { operation: \"divide\", number1: 3, number2: 12 },\n",
            "    id: \"call_BDuJv8QkDZ7N7Wsd6v5VDeVa\"\n",
            "  }\n",
            "]\n"
          ]
        }
      ],
      "source": [
        "import { HumanMessage, AIMessage, ToolMessage } from \"@langchain/core/messages\";\n",
        "\n",
        "const res = await llmWithTools.invoke([\n",
        "  new HumanMessage(\"What is 333382 🦜 1932?\"),\n",
        "  new AIMessage({\n",
        "    content: \"\",\n",
        "    tool_calls: [{\n",
        "      id: \"12345\",\n",
        "      name: \"calulator\",\n",
        "      args: {\n",
        "        number1: 333382,\n",
        "        number2: 1932,\n",
        "        operation: \"divide\",\n",
        "      }\n",
        "    }]\n",
        "  }),\n",
        "  new ToolMessage({\n",
        "    tool_call_id: \"12345\",\n",
        "    content: \"The answer is 172.558.\"\n",
        "  }),\n",
        "  new AIMessage(\"The answer is 172.558.\"),\n",
        "  new HumanMessage(\"What is 3 🦜 12\")\n",
        "]);\n",
        "console.log(res.tool_calls);"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Binding model-specific formats (advanced)\n",
        "\n",
        "Providers adopt different conventions for formatting tool schemas. For instance, OpenAI uses a format like this:\n",
        "\n",
        "- `type`: The type of the tool. At the time of writing, this is always \"function\".\n",
        "- `function`: An object containing tool parameters.\n",
        "- `function.name`: The name of the schema to output.\n",
        "- `function.description`: A high level description of the schema to output.\n",
        "- `function.parameters`: The nested details of the schema you want to extract, formatted as a [JSON schema](https://json-schema.org/) object.\n",
        "\n",
        "We can bind this model-specific format directly to the model if needed. Here's an example:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "AIMessage {\n",
              "  lc_serializable: \u001b[33mtrue\u001b[39m,\n",
              "  lc_kwargs: {\n",
              "    content: \u001b[32m\"\"\u001b[39m,\n",
              "    tool_calls: [\n",
              "      {\n",
              "        name: \u001b[32m\"calculator\"\u001b[39m,\n",
              "        args: { operation: \u001b[32m\"multiply\"\u001b[39m, number1: \u001b[33m119\u001b[39m, number2: \u001b[33m8\u001b[39m },\n",
              "        id: \u001b[32m\"call_pBlKOPNMRN4AAMkPaOKLLcyj\"\u001b[39m\n",
              "      }\n",
              "    ],\n",
              "    invalid_tool_calls: [],\n",
              "    additional_kwargs: {\n",
              "      function_call: \u001b[90mundefined\u001b[39m,\n",
              "      tool_calls: [\n",
              "        {\n",
              "          id: \u001b[32m\"call_pBlKOPNMRN4AAMkPaOKLLcyj\"\u001b[39m,\n",
              "          type: \u001b[32m\"function\"\u001b[39m,\n",
              "          function: \u001b[36m[Object]\u001b[39m\n",
              "        }\n",
              "      ]\n",
              "    },\n",
              "    response_metadata: {}\n",
              "  },\n",
              "  lc_namespace: [ \u001b[32m\"langchain_core\"\u001b[39m, \u001b[32m\"messages\"\u001b[39m ],\n",
              "  content: \u001b[32m\"\"\u001b[39m,\n",
              "  name: \u001b[90mundefined\u001b[39m,\n",
              "  additional_kwargs: {\n",
              "    function_call: \u001b[90mundefined\u001b[39m,\n",
              "    tool_calls: [\n",
              "      {\n",
              "        id: \u001b[32m\"call_pBlKOPNMRN4AAMkPaOKLLcyj\"\u001b[39m,\n",
              "        type: \u001b[32m\"function\"\u001b[39m,\n",
              "        function: {\n",
              "          name: \u001b[32m\"calculator\"\u001b[39m,\n",
              "          arguments: \u001b[32m'{\"operation\":\"multiply\",\"number1\":119,\"number2\":8}'\u001b[39m\n",
              "        }\n",
              "      }\n",
              "    ]\n",
              "  },\n",
              "  response_metadata: {\n",
              "    tokenUsage: { completionTokens: \u001b[33m24\u001b[39m, promptTokens: \u001b[33m85\u001b[39m, totalTokens: \u001b[33m109\u001b[39m },\n",
              "    finish_reason: \u001b[32m\"tool_calls\"\u001b[39m\n",
              "  },\n",
              "  tool_calls: [\n",
              "    {\n",
              "      name: \u001b[32m\"calculator\"\u001b[39m,\n",
              "      args: { operation: \u001b[32m\"multiply\"\u001b[39m, number1: \u001b[33m119\u001b[39m, number2: \u001b[33m8\u001b[39m },\n",
              "      id: \u001b[32m\"call_pBlKOPNMRN4AAMkPaOKLLcyj\"\u001b[39m\n",
              "    }\n",
              "  ],\n",
              "  invalid_tool_calls: []\n",
              "}"
            ]
          },
          "execution_count": 1,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import { ChatOpenAI } from \"@langchain/openai\";\n",
        "\n",
        "const model = new ChatOpenAI({ model: \"gpt-4o\" });\n",
        "\n",
        "const modelWithTools = model.bind({\n",
        "  tools: [{\n",
        "    \"type\": \"function\",\n",
        "    \"function\": {\n",
        "      \"name\": \"calculator\",\n",
        "      \"description\": \"Can perform mathematical operations.\",\n",
        "      \"parameters\": {\n",
        "        \"type\": \"object\",\n",
        "        \"properties\": {\n",
        "          \"operation\": {\n",
        "            \"type\": \"string\",\n",
        "            \"description\": \"The type of operation to execute.\",\n",
        "            \"enum\": [\"add\", \"subtract\", \"multiply\", \"divide\"]\n",
        "          },\n",
        "          \"number1\": {\"type\": \"number\", \"description\": \"First integer\"},\n",
        "          \"number2\": {\"type\": \"number\", \"description\": \"Second integer\"},\n",
        "        },\n",
        "        \"required\": [\"number1\", \"number2\"],\n",
        "      },\n",
        "    },\n",
        "  }],\n",
        "});\n",
        "\n",
        "await modelWithTools.invoke(`Whats 119 times 8?`);"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "This is functionally equivalent to the `bind_tools()` calls above."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Next steps\n",
        "\n",
        "Now you've learned how to bind tool schemas to a chat model and to call those tools. Next, check out some more specific uses of tool calling:\n",
        "\n",
        "- [Building tool-using chains and agents](/docs/how_to/#tools)\n",
        "- [Getting structured outputs from models](/docs/how_to/structured_output/)"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Deno",
      "language": "typescript",
      "name": "deno"
    },
    "language_info": {
      "file_extension": ".ts",
      "mimetype": "text/x.typescript",
      "name": "typescript",
      "nb_converter": "script",
      "pygments_lexer": "typescript",
      "version": "5.3.3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}
